What is claimed is: 



1 1 . An automated testing method for use in a communications network with a 

2 plurality of different clocks, the method comprising: 

3 storing clock error information for said plurality of different clocks; 

4 monitoring a plurality of network points to detect events; 

5 generating a set of test results including detected events and corresponding event 

6 times, at least some of said event times having been obtained from timing information 

7 corresponding to different clocks; 

8 processing said set of test results to generate set of timing corrected test results, 

9 said processing including modifying at least some event times as a function of said stored 

1 0 clock error information; 

1 1 comparing events and corresponding event times included in the set of timing 

12 corrected test results to expected events and corresponding expected event times included 

13 in a set of expected test results; and 

14 generating a set of test result information including information indicating 

15 differences between the set of timing corrected test results and expected test results. 

1 2. The method of claim 1, further comprising: 

2 storing a set of relative timing test results, said set of relative timing test results 

3 including expected events and relative event times corresponding to the expected events, 

4 said relative times being expressed relative to at least one of a test stimulus time and a 

5 preceding expected event time; and 

6 processing said set of relative timing test results using stimulus time information 

7 to produce said set of expected test results, said expected event times being generated as a 

8 function of said stimulus time information and said relative times. 

1 3. The method of claim 2, wherein said step of generating as set of test result 

2 information includes identifying, as a function of said comparing, detected unexpected 

3 events and undetected expected events. 
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1 4. The method of claim 3, wherein detected events included in said set of timing 

2 corrected test results are identified as detected unexpected events if the detected event 

3 and the timing associated with the detected event do not match an expected event and the 

4 timing associated with said expected event. 

1 5. The method of claim 3, wherein undetected expected events are events included 

2 in said set of expected test results which are not detected in the set of time corrected test 

3 results at the expected time. 

1 6. The method of claim 5, wherein said step of generating as set of test result 

2 information includes identifying, as a function of said comparing: 

3 expected undetected events which include events in the expected test results 

4 which are not found in said set of time corrected test results. 

1 7. The method of claim 6, where said set of expect test result includes lists of 

2 expected events arranged based on causal dependency; and 

3 wherein said step of generating a set of test result information includes listing at 

4 most one undetected expected event corresponding to each list of expected events 

5 arranged based on causal dependency included in said set of expected test results. 

1 8. The method of claim 1, wherein said stored clock error information includes 

2 information relating to clocks included in different routing devices, wherein said set of 

3 test results includes detected events corresponding to different network links and wherein 

4 time information associated with detected events corresponding to different network links 

5 is obtained from clocks included in different routing devices 

1 9. The method of claim 8, wherein said monitored network points correspond to 

2 different points in a system including at least four active network routing devices; at least 

3 two monitored network points corresponding to the output of two different network 

4 routing devices. 
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1 1 0. The method of claim 9, wherein said four active network routing devices are 

2 telephone switches. 

1 11. The method of claim 9, wherein said four active network routing devices are IP 

2 packet routers. 

1 12. A testing apparatus for performing communications system testing, said 

2 communications system including a plurality of routing devices having different time 

3 clocks, the apparatus comprising: 

4 timing information corresponding to at least some of said clocks indicating timing 

5 differences between said at least some clocks; 

6 relative timing expected test results including expected events and relative 

7 expected event times, each of said relative expected event times being expressed relative 

8 to a test stimulus or another expected event time; 

9 an expected results translator module for translating the relative timing 

10 information included in said relative timing expected results to actual times as a function 

11 of at least one test stimulus time, said expected results translator module producing 

12 timing corrected expected results; 

13 raw test results retrieved from monitoring points in said communications system, 

14 said raw test results including detected events and detected event times obtained from 

15 different clocks ; 

16 a timing correction module for processing said raw test results as a function of 

17 said timing information to generate a set of detected timing corrected results; and 

18 an analysis module for processing said timing corrected expected results and said 

19 detected timing corrected results to generate a set of test results including information 

20 identifying undetected expected results and detected unexpected results, said undetected 

21 expected results and said detected unexpected results being determined as a function of a 

22 comparison between said timing corrected expected results and said detected timing 

23 corrected results. 
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1 13. The apparatus of claim 1 2, further comprising: 

2 a report generation module for classifying detected timing corrected events as a 

3 function of the comparison between said detected unexpected results and said timing 

4 corrected expected results, said classifying including classifying events as detected 

5 unexpected events when said detected timing corrected events do not match an expected 

6 event. 

1 14. The apparatus of claim 13, wherein said timing information includes clock error 

2 information for a plurality of different clocks included in said communications system. 

1 15. The apparatus of claim 14, further comprising: 

2 means for detecting events at a plurality of different test points in said 

3 communications system, said test points including a point at which said stimulus can be 

4 monitored and a point separated from said stimulus point by at least one routing device. 

1 16. The apparatus of claim 15, wherein the communication system is a telephone 

2 communication system including at least four signal transfer points, said plurality of 

3 different test points including inputs and outputs of at least three different signal transfer 

4 points. 

1 17. The apparatus of claim 15, wherein the communications system is an IP packet 

2 routing system and wherein said plurality of different test points includes inputs and 

3 outputs of at least three different routers. 

1 1 8. The apparatus of claim 15, further comprising: 

2 stored test stimulus information; and 

3 means for introducing test stimulus including multiple link failures into said 

4 communications system according to said stored test stimulus information. 
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1 19. An automated test system for testing communications networks including a 

2 plurality of nodes, each of said nodes including a different clock, the test system 

3 including: 

4 network node clock information including information suitable for synchronizing 

5 time values received from said multiple clocks; 

6 a set of detected events and detected event times; 

7 means for correcting said detected event times as a function of network node 

8 clock information to synchronize the time values obtained from different network node 

9 clocks; and 

10 means for comparing the detected events and corresponding corrected event times 

1 1 to expected events and corresponding expected event times to identify unexpected 

12 detected events. 

1 20. The system of claim 19 wherein said means for comparing further identifies 

2 undetected expected events. 

1 21. The system of claim 20, wherein said detected events include events 

2 corresponding to a link failure, said test system further comprising: 

3 means for monitoring the failed link and a communications link separated from 

4 the failed link by at least two network nodes including routing functionality. 
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